IMAGE PROCESSING SYSTEM, PROJECTOR, PROGRAM, INFORMATION 
STORAGE MEDIUM AND IMAGE PROCESSING METHOD 



Japanese Patent Application No. 2003-91720, filed on March 28, 2003 is 
5 hereby incorporated by reference in its entirety. 

BACKGROUD OF THE INVENTION 
The present invention relates to an image processing system, projector, 
program, information storage medium and image processing method which can detect a 
10 sensed projected image based on sensed information. 

If an image is displayed in an image display device such as a projector for a 
long term, the image will gradually deteriorate due to color non-uniformity occurred 
therein (including color non-uniformity from luminance non-imiformity). 

For example, in a store or exhibition ground, an image may continuously be 
15 displayed on a screen through a projector for a long term. In such a case, an original 
display effect may be lost by color non-uniformity occurred in the image due to 
deterioration of a liquid crystal panel or polarizing plate from the long-term projection 
of the image. 

In view of such a problem, for example, Japanese Patent Laid-Open No. 
20 2002-140060 has proposed an image processing system which can more adequately 
correct color non-uniformity due to deterioration v^th time by detecting a sensed 
projected image based on sensed information and using the luminance value within the 
sensed projected image included in the sensed information. 

However, Japanese Patent Laid-Open No. 2002-140060 does not particularly 
25 describe a technique for causing the coordinates of the sensed projected image in the 
sensing area to be precisely associated with the coordinates of the spatial light 
modulator. 
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In particular, if there is a distortion in the image, it is important that the 
coordinates of the sensed projected image in the sensing area is precisely associated 
with the coordinates of the spatial light modulator. 



5 BRIEF SUHMARY OF THE INVENTION 

The present invention is made in view of the above-mentioned problem and its 
object is to provide an image processing system, projector, program, information 
storage medium and image processing method which can cause the coordinates of the 
sensed projected image in the sensing area to be precisely associated with the 
10 coordinates of the spatial light modulator. 

To solve the above problem, according to a first aspect of the present invention, 
there is provided an image processing system comprising: 

image projection means for projecting at least two different colored calibration 
images at different points of time; 
15 sensing means for sensing each of the projected calibration images to output 

sensed information; 

ratio information generating means for computing a ratio of image signal 
values or luminance values for each pixel in a sensing area obtained by sensing the 
calibration images, based on the sensed information to generate ratio information; 

20 edge point detecting means for searching the ratio information to find a first 

value which represents a ratio of image signal values or luminance values among pixel 
regions each having a predetermined number of pixels in the sensing area, and for 
detecting edge points of a sensed projected image, based on part of the pixel regions 
each having the first value equal to or larger than a first predetermined value; 

25 pixel block image information generating means for converting sensed 

information of an area defined by the detected edge points into pixel block image 
information representing a ratio of image signal values or luminance values for each 
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pixel block including one or more pixels, based on the sensed information and the 
detected edge points; and 

correction means for correcting an image signal based on the pixel block image 
information, 

5 wherein the image projection means projects an image based on the corrected 

image signal. 

According to a second aspect of the present invention, there is provided an 
image processing system comprising: 

image projection section which projects at least two different colored 
10 calibration images at different points of time; 

sensing section which senses each of the projected calibration images to output 
sensed information; 

ratio information generating section which computes a ratio of image signal 
values or luminance values for each pixel in a sensing area obtained by sensing the 
15 calibration images, based on the sensed information to generate ratio information; 

edge point detecting section which searches the ratio information to find a first 
value which represents a ratio of image signal values or luminance values among pixel 
regions each having a predetermined number of pixels in the sensing area, and detects 
edge points of a sensed projected image, based on part of the pixel regions each having 
20 the first value equal to or larger than a first predetermined value; 

pixel block image information generating section which converts sensed 
information of an area defined by the detected edge points into pixel block image 
information representing a ratio of image signal values or luminance values for each 
pixel block including one or more pixels, based on the sensed information and the 
25 detected edge points; and 

correction section which corrects an image signal based on the pixel block 
image information. 
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wherein the image projection section projects an image based on the corrected 
image signal. 

According to a third aspect of the present invention, there is provided a 
projector comprising: 

5 image projection means for projecting at least two different colored calibration 

images at diflferent points of time; 

sensing means for sensing each of the projected calibration images to output 
sensed information; 

ratio information generating means for computing a ratio of image signal 
10 values or luminance values for each pixel in a sensing area obtained by sensing the 
calibration images, based on the sensed information to generate ratio information; 

edge point detecting means for searching the ratio information to find a first 
value which represents a ratio of image signal values or luminance values among pixel 
regions each having a predetermined number of pixels in the sensing area, and for 
15 detecting edge points of a sensed projected image, based on part of the pixel regions 
each having the first value equal to or larger than a first predetermined value; 

pixel block image information generating means for converting sensed 
information of an area defined by the detected edge points into pixel block image 
information representing a ratio of image signal values or luminance values for each 
20 pixel block including one or more pixels, based on the sensed information and the 
detected edge points; and 

correction means for correcting an image signal based on the pixel block image 
information, 

wherein the image projection means projects an image based on the corrected 
25 image signal. 

According to a fourth aspect of the present invention, there is provided a 
projector comprising: 
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image projection section which projects at least two different colored 
calibration images at different points of time; 

sensing section which senses each of the projected calibration images to output 
sensed information; 

5 ratio information generating section which computes a ratio of image signal 

values or luminance values for each pixel in a sensing area obtained by sensing the 
calibration images, based on the sensed information to generate ratio information; 

edge point detecting section which searches the ratio information to find a first 
value which represents a ratio of image signal values or limiinance values among pixel 
10 regions each having a predetermined number of pixels in the sensing area, and detects 
edge points of a sensed projected image, based on part of the pixel regions each having 
the first value equal to or larger than a first predetermined value; 

pixel block image information generating section which converts sensed 
information of an area defined by the detected edge points into pixel block image 
15 information representing a ratio of image signal values or luminance values for each 
pixel block including one or more pixels, based on the sensed information and the 
detected edge points; and 

correction section which corrects an image signal based on the pixel block 
image information, 

20 wherein the image projection section projects an image based on the corrected 

image signal. 

According to a fifth aspect of the present invention, there is provided a 
computer-readable program causing a computer to function as: 

image projection means for projecting at least two different colored calibration 
25 images at different points of time; 

sensing means for sensing each of the projected calibration images to output 
sensed information; 
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ratio information generating means for computing a ratio of image signal 
values or luminance values for each pixel in a sensing area obtained by sensing the 
calibration images, based on the sensed information to generate ratio information; 

edge point detecting means for searching the ratio information to find a first 
5 value which represents a ratio of image signal values or luminance values among pixel 
regions each having a predetermined number of pixels in the sensing area, and for 
detecting edge points of a sensed projected image, based on part of the pixel regions 
each having the first value equal to or larger than a first predetermined value; 

pixel block image information generating means for converting sensed 
10 information of an area defined by the detected edge points into pixel block image 
information representing a ratio of image signal values or luminance values for each 
pixel block including one or more pixels, based on the sensed information and the 
detected edge points; and 

correction means for correcting an image signal based on the pixel block image 
15 information, 

wherein the image projection means projects an image based on the corrected 
image signal. 

According to a sixth aspect of the present invention, there is provided an 
information storage medium storing a computer-readable program which causes a 
20 computer to function as: 

image projection means for projecting at least two different colored calibration 
images at different points of time; 

sensing means for sensing each of the projected calibration images to output 
sensed information; 

25 ratio information generating means for computing a ratio of image signal 

values or luminance values for each pixel in a sensing area obtained by sensing the 
calibration images, based on the sensed information to generate ratio information; 
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edge point detecting means for searching the ratio information to find a first 
value which represents a ratio of image signal values or luminance values among pixel 
regions each having a predetermined number of pixels in the sensing area, and for 
detecting edge points of a sensed projected image, based on part of the pixel regions 
5 each having the first value equal to or larger than a first predetermined value; 

pixel block image information generating means for converting sensed 
information of an area defined by the detected edge points into pixel block image 
information representing a ratio of image signal values or Ixmiinance values for each 
pixel block including one or more pixels, based on the sensed information and the 
10 detected edge points; and 

correction means for correcting an image signal based on the pixel block image 
information, 

wherein the image projection means projects an image based on the corrected 
image signal. 

15 According to a seventh aspect of the present invention, there is provided an 

image processing method comprising: 

sequentially projecting a plurality of monochromatic calibration images of 
different colors; 

sensing the projected calibration images and outputting sensed information; 
20 computing a ratio of image signal values or luminance values for each pixel in 

a sensing area obtained by sensing the calibration images, based on the sensed 
information; 

generating ratio information for the sensing area; 

searching the ratio information to find a first value which represents a ratio of 
25 image signal values or luminance values among pixel regions each having a 
predetermined number of pixels in the sensing area, and detecting edge points of a 
sensed projected image, based on part of the pixel regions having the first value equal to 
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or larger than a first predetermined value; 

converting sensed information of an area defined by the detected edge points 
into pixel block image information representing a ratio of image signal values or 
luminance values for each pixel block including one or more pixels, based on the sensed 
5 information and the detected edge points; 

correcting an image signal to correct at least one of distortion, color and 
brightness in the image, based on the pixel block image information; and 

projecting an image based on the corrected image signal. 

The image processing system and others of the present invention can more 

10 precisely and effectively detect edge points in a sensed projected image than the 
conventional technique of detecting edge points through filtering, by using ratio 
information provided when the ratio of image signal values or luminance values of the 
associated pixels in the sensing area is computed. Thus, the image processing system 
and others can cause the coordinates of the sensed projected image in the sensing area to 

15 be precisely associated with the coordinates of the spatial light modulator. 

The image processing system and others can more effectively perform the 
image processing because they use the ratio rather than the differential value and do not 
require any change of the first predetermined value which is a determination criterion 
for edge points in the sensed projected image. 

20 Moreover, an accurate correction can be made for each pixel block even if the 

sensed projected image is divided into pixel blocks for correcting image signals, since 
the sensed projected image in the sensing area can be accurately detected. Thus, a 
correct image can be displayed. 

The number of pixel blocks in the pixel block image information may be the 

25 number of pixel blocks according to the processing of the correction means. 
Furthermore, the pixel area or pixel block may correspond, for example, to one pixel, a 
plurality of pixels, one pixel line, a plurality of pixel lines, a rectangular area configured 



by a plurality of pixels or the like. It is not necessary that the pixel area and pixel block 
have the same unit. For example, a pixel area may includes one pixel and a pixel block 
may be a rectangular area including a plurality of pixels. 

In the image processing system, projector, program, and information storage 
5 medium, the ratio information generating means may include means for detecting a 
rectangular region defined by the pixel regions each having the first value equal to or 
larger than the first predetermined value as a temporary sensed projected image by 
searching the ratio information for the first value in vertical and horizontal directions 
from comer points of an area corresponding to the sensing area represented by the ratio 

10 information; an image projected by the image projection means may be a quadrangle; 
and the edge point detecting means may detect comer points at four comers of the 
sensed projected image as the edge points of the sensed projected image, based on the 
temporary sensed projected image. 

The image processing method may further comprise: 

15 detecting a rectangular region defined by the pixel regions each having the first 

value equal to or larger than the first predetermined value as a temporary sensed 
projected image by searching the ratio information for the first value in vertical and 
horizontal directions from comer points of an area corresponding to the sensing area 
represented by the ratio information; and 

20 detecting comer points at four comers of the sensed projected image as the 

edge points of the sensed projected image, based on the temporary sensed projected 
image. 

Thus, when an ordinary rectangular image is projected, the image processing 
system and others can precisely and effectively detect the four comer points in the 
25 sensed projected image by simply detecting a temporary sensed projected image and 
then detecting an actual sensed projected image based on the temporary sensed 
projected image. 
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In the image processing system, projector, program and information storage 

medium, 

when the number of detected comer points is three or less in detecting comer 
points at four comers of the sensed projected image, the edge point detecting means 
5 may detect other comer points by using a predetermined method; and 
the predetermined method may include: 

counting the numbers of pixels from an edge pixel to a pixel having the ratio 
represented by the ratio information equal to or larger than a second predetermined 
value in each pixel line of the temporary sensed projected image; 
10 computing a changing rate of a ratio of the number of pixels in every two 

adjacent pixel lines of the temporary sensed projected image; and 

detecting pixels having the ratio represented by the ratio information equal to 
or larger than the second predetermined value as the other comer points in a pixel line 
having the changing rate equal to or larger than a third predetermined value. 
15 In the image processing method, 

when the nxmiber of detected comer points is three or less in detecting comer 
points at four comers of the sensed projected image, other comer points may be 
detected by a predetermined method; and 

the predetermined method may include: 
20 counting the numbers of pixels from an edge pixel to a pixel having the ratio 

represented by the ratio information equal to or larger than a second predetermined 
value in each pixel line of the temporary sensed projected image; 

computing a changing rate of a ratio of the number of pixels in every two 
adjacent pixel lines of the temporary sensed projected image; and 
25 detecting pixels having the ratio represented by the ratio information equal to 

or larger than the second predetermined value as the other comer points in a pixel line 
having the changing rate equal to or larger than a third predetermined value. 

10 



Thus, the image processing system and others can detects correct comer points 
through different searching methods even if only three or less comer points are detected 
in detecting four comer points in the sensed projected image. Thus, even if the image is 
distorted into various different shapes, the image processing system and others can 
5 precisely and effectively detect the four comer points in the sensed projected image. 
Therefore, the coordinates of the sensed projected image in the sensing area can 
precisely be associated with the coordinates of the spatial light modulator. 

In connection with this, the first and second predetermined values may be 
identical with or different from each other. 

10 In the image processing system, projector, program and information storage 

medium, the correction means may include: distortion correction means for correcting 
an image signal to correct a distortion in an image based on the pixel block image 
information when a predetermined calibration image is projected; and color 
non-uniformity correction means for correcting an image signal to correct color 

15 non-uniformity based on the pixel block image information when a predetermined 
calibration image is projected; and the image projection means may project an image 
based on the corrected image signal. 

Since the image processing system and others can precisely detect the sensed 
projected image in the sensing area, they can precisely correct the distortion and color 

20 non-uniformity in the image for each pixel block even if the sensed projected image is 
divided into pixel blocks for correcting image signals. In such a manner, a correct image 
can be displayed. 

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING 
25 FIG. 1 is a diagram schematically showing an image projected according to one 

embodiment of the present invention. 

FIG 2 is a diagram showing a sensing area according to one embodiment of the 
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present invention. 

FIG 3 is a functional block diagram of a projector according to one 
embodiment of the present invention. 

FIG. 4 is a block diagram showing hardware configuration of a projector 
5 according to one embodiment of the present invention. 

FIG 5 is a flow chart for illustrating image processing according to one 
embodiment of the present invention. 

FIG 6 A is a diagram schematically showing a temporary sensed projected 
image when three comer points have been detected; and FIG 6B is a diagram 
10 schematically showing a temporary sensed projected image when two comer points 
have been detected. 

FIG 7 is a diagram schematically showing a pixel block image according to 
one embodiment of the present invention. 

15 DETAILED DESCRIPTION OF THE EMBODIMENT 

The present invention will now be described in connection with a projector 
having an image processing system with reference to the drawing. It is to be understood 
that an embodiment shown in the following is not intended to limit the subject matter of 
the invention as described in the accompanying claims and that all the components 

20 shown in such an embodiment is not necessarily essential for practicing the invention 
defined by the accompanying claims. 

The Whole System 

FIG 1 is a diagram schematically showing an image projected according to one 
25 embodiment of the present invention. 

A projector 20 projects an image onto a screen 10. Thus, a projected image 12 
is formed on the screen 10. 
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When an image is continuously displayed through a projection type image 
display device such as the projector 20 for a long term, color non-uniformity (including 
color non-uniformity due to luminance non-uniformity) will gradually occur in the 
projected image 12 due time degradation as in its optical system or the like, 
5 In addition, the image appearance in the projected image 12 will vary due to an 

ambient light 80 such as sxmlight and illuminating light. 

Furthermore, the projected image 12 may be distorted depending on an angle 
included between the screen 10 and the optical axis of the projected light from the 
projector 20. 

10 To correct such color non-uniformity and distortion in the projected image 12, 

the projector 20 according to this embodiment has a sensor 60 which functions as 
sensing means. The sensor 60 picks up a screen region including the projected image 
12. 

It is assumed in this embodiment that the sensor 60 is mounted in a so-called 
15 remote controller and configured as a component separate from the projector 20. As a 
user activates a sensing button on the remote controller, the projected image 12 on the 
screen 10 is sensed through the sensor 60, The projector 20 receives, from the sensor 60, 
sensed information which represents an image signal value for each pixel in the sensing 
area or a luminance value transformed from the image signal value. 
20 FIG 2 is a diagram showing a sensing area 210 according to one embodiment 

of the present invention. 

Referring to FIG. 2, the sensing area 210 is shown to be a rectangular region 
ABCD including a predetermined number of pixels. A sensed projected image 220 
representing the actual shape of a projected image is shown to be a region surrounded 
25 by four points I, J, K and L. A rectangular region EFGH adjoining to the region IJKL 
forms a temporary sensed projected image 230. 

In this embodiment, the projector 20 computes and stores a ratio for each pixel 
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between an image signal value or luminance value in the sensing area 210 when a 
white-colored monochromatic image is projected and an image signal value or 
luminance value in the sensing area 210 when a black-colored monochromatic image is 
projected or not projected. 
5 The projector 20 compares the sum of these ratios with a first predetermined 

value for each of predetermined pixel regions (each including one pixel, one line, a 
rectangular region configured by a plurality of pixels or the like) which are taken in the 
inward direction from each comer point in the sensing area 210. 

The projector 20 then detects, as the temporary sensed projected image 230, a 
10 rectangular region surrounded by pixel regions in which the sum of ratios is judged to 
be equal to or larger than the first predetermined value when the search is performed in 
four vertical and horizontal directions. 

Furthermore, the projector 20 compares the ratio for each of the pixels disposed 
on the respective sides of the temporary sensed projected image 230 extending from the 
15 four comers thereof (as the case may be, within the interior of the temporary sensed 
projected image 230) with a second predetermined value. The projector 20 then detects 
the sensed projected image 220 which includes comer points consisting of pixels each 
of which has its ratio equal to or larger than the second predetermined value. 

In such a manner, the projector 20 according to this embodiment can more 
20 effectively detect the sensed projected image 220 by detecting it after the temporary 
sensed projected image 230 has been detected simply since it is unnecessary that the 
sensed projected image 220 is detected through the filtering as in the prior art. 

In this embodiment, furthermore, the projector 20 causes the projected image 
12 to be associated with the display device thereof by generating a predetermined pixel 
25 block image based on the sensed projected image 220 thus detected. 

Thus, the projector 20 can more precisely correct the distortion or color in the 
projected image. 
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The functional blocks of the projector 20 for implementing such a feature will 
be described blow. 

FIG. 3 is a functional block diagram of the projector 20 according to one 
embodiment of the present invention. 
5 The projector 20 comprises an input signal processing section 110 for 

converting analog G- and B-signals (Rl, Gl, Bl) from a PC (Personal Computer) 
into digital G- and B-signals (R2, G2, B2); a color converting section 120 for 
converting the digital R-, G- and B-signals (R2, G2, B2) into digital R-, G- and 
B-signals (R3, G3, B3) to correct the color and brightness of the image; a color 

10 non-xmiformity correcting section 130 for converting the digital R-, G- and B-signals 
(R3, G3, B3) into digital R-, G- and B-signals (R4, G4, B4) to correct color 
non-uniformity; an output signal processing section 140 for converting the digital R-, G- 
and B-signals (R4, G4, B4) into analog R-, G- and B-signals (R5, G5, B5); and an 
image projecting section 190 for projecting an image based on the analog R-, G- and 

15 B-signals. 

The input signal processing section 110 comprises an image distortion 
correcting section 112 for correcting an image distortion (including a trapezoidal 
distortion. Keystone distortion and the like) against the digital converted R-, G- and 
B-signals. 

20 The color converting section 120 comprises a 3D-LUT storage section 122 for 

storing a three-dimensional lookup table (3D-LUT) for image color correction, and a 
ID-LUT storage section 124 for storing one-dimensional lookup table (ID-LUT) for 
image brightness correction. 

The image projecting section 190 comprises a spatial light modulator 192, a 

25 drive section 194 for driving the spatial light modulator 192, a light source 196 and a 
lens 198. 

The drive section 194 drives the spatial light modulator 192 based on the image 
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signals from the signal output section 140. The image projecting section 190 projects 
light from the light source 196 through the spatial light modulator 192 and lens 198. 

The projector 20 further comprises a calibration signal generating section 172 
for generating image signals used to display a calibration image; an sensed information 
5 storage section 184 for temporarily storing the sensed information and the like from the 
sensor 60; a ratio information generating section 152 for generating ratio information 
based on the sensed information and for detecting the temporary sensed projected image 
230; an endpoint detecting section 154 for detecting comer points in the sensed 
projected image 220 based on the temporary sensed projected image 230; and a pixel 

10 block image information generating section 156 for generating a pixel block image 
based on the comer points. 

The projector 20 further comprises an image distortion correction computing 
section 162 for computing an amount of image distortion correction for the image 
distortion correcting section 112 based on the pixel block image; a color non-uniformity 

15 correction computing section 164 for computing an amount of color non-uniformity 
correction for the color non-uniformity correcting section 130 based on the pixel block 
image; and a pixel sensing setting section 186 for generating a control signal used to 
control the sensor 60 so that an image is sensed with the number of pixels according to 
the requirements from the image distortion correction computing section 162 and color 

20 non-imiformity correction computing section 164. 

The sending and receiving of information between the sensor 60 and the sensed 
information storage section 184 and pixel sensing setting section 186 may be made by 
any radio communication such as blue tooth and infrared rays or may be made by any 
wire communication using LAN cables or the like according to any communication 

25 protocol based on TCP/IP or the like. 

The hardware for implementing each section in the projector 20 may use the 
followings. 
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FIG 4 is a block diagram showing hardware configuration of the projector 20 
according to one embodiment of the present invention. 

For example, the input signal processing section 110 may be implemented by 
an A/D converter 930 or the like; the sensed information storage section 184 may be 
5 implemented by a RAM 950 or the like; the color non-uniformity correcting section 
130, ratio information generating section 152, endpoint detecting section 154, pixel 
block image information generating section 156 and calibration signal generating 
section 172 may be implemented by an image processing circuit 970 or the like; the 
image distortion correction computing section 162 and color non-uniformity correction 
10 computing section 164 may be implemented by CPU 910 or the like; the color 
converting section 120 may be implemented by the image processing circuit 970, RAM 
950, CPU 910 or the like; the output signal processing section 140 may be implemented 
by a D/A converter 940 or the like; the spatial light modulator 192 may be implemented 
by a liquid crystal panel 920 or the like; and the drive section 194 may be implemented 
15 by the use of a ROM 960 or the like for storing a liquid crystal light valve driver for 
driving the liquid crystal panel 920. 

These components are configured to mutually deliver the information 
therebetween through. a system bus 980. 

These components may be implemented in a hardware manner or in a software 
20 manner such as drivers. 

Furthermore, the computer may implement the functions of the ratio 
information generating section 152 and the like according to a program stored in and 
read out of an information storage medium 900. 

Such an information storage mediimi 900 may be accomplished, for example, 
25 by CD-ROM, DVD-ROM, ROM, RAM, HDD or the like through either of the contact 
or non-contact type reading mode. 

Rather than the information storage medium 900, the aforementioned functions 
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can be implemented by downloading a program or the like for implementing them from 
a host device or the like through a transmission channel. 

Flow of Image Processing 
5 The flow of image processing by use of these parts will be described below. 

FIG 5 is a flow chart for illustrating image processing according to one 
embodiment of the present invention. 

First of all, the projector 20 projects a plurality of different monochromatic 
calibration images (step SI). More particularly, if an image distortion is to be corrected, 
10 the projector 20 projects white- and black-colored calibration images. If the color and 
brightness in an image are to be corrected, the projector 20 projects R, G and B 
calibration images each of which gray scales has been changed by a predetermined 
gradation imit, in addition to the aforementioned calibration images. 

First of all, the image projecting section 190 projects a first calibration image 
15 onto the screen 10 (step SI). 

The sensor 60 changes the setting so that an image will be sensed with a 
predetermine number of pixels, based on control information transmitted firom the pixel 
sensing setting section 186 in the projector 20, and captures the first calibration image 
projected onto the screen 10 by means of the image projecting section 190 when the 
20 user activates the button (step S2). 

In such a manner, the sensed information can be processed more effectively for 
the purpose since the amoimt of information can be controlled by adjusting the number 
of pixels to be sensed. 

The sensor 60 then sends the sensed information to the projector 20 while the 
25 sensed information storage section 184 stores the sensed information (step S3). 

The calibration signal generating section 172 judges whether or not the 
projection of all the calibration images has been done (step S4). If not so, the projector 
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20 repeatedly carries out the steps SI to S3. 

In such a manner, the sensed information storage section 1 84 stores the sensed 
information relating to all the calibration images. The ratio information generating 
section 152 then computes a ratio for each pixel (e.g., R-signal value or the like) based 
5 on the sensed information for the white- and black-colored calibration images which 
have been stored in the sensed information storage section 184, and detects a 
rectangular-shaped temporary sensed projected image 230 based on the ratio (step S5). 
The ratio information generating section 152 causes the sensed information storage 
section 1 84 to store the ratio information which represents the ratio for each pixel in the 

10 sensed information. 

The ratio may be (the luminance value of the white-colored calibration 
image/the luminance value of the black-colored calibration image- 1) or (the luminance 
value of the white-colored calibration image/the luminance value of the black-colored 
calibration image. In this embodiment, it is supposed that the ratio is the luminance 

15 value of the white-colored calibration image/the luminance value of the black-colored 
calibration image. 

The ratio information generating section 152 determines whether or not the 
sum of ratios in the respective pixel lines arranged inwardly from the respective sides of 
the sensing area 210 is equal to or larger than a predetermined value (e.g., 2.0 * the 

20 number of pixels forming the pixel line). For example, the ratio information generating 
section 152 may search each pixel line downwardly from a side AB in FIG. 2 and 
terminate the downward search at a point of time when the aforementioned sum 
becomes equal to or larger than the predetermined value in the pixel lines on another 
side EF. The ratio information generating section 1 52 similarly performs the searches in 

25 the rightward direction from a side AD, in the leftward direction from a side BC and in 
the upward direction from a side CD and then detects the temporary sensed projected 
image 230 surrounded by the four sides at each of which the sum is equal to or larger 
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than the predetermined value. 

The ratio information generating section 152 may have changed the luminance 
value at which the ratio of luminance values in the corresponding pixels is less than a 
predetermined value (e.g., 2.0) into zero as pre-processing prior to the detection of the 
5 temporary sensed projected image 230 and caused it to be stored in the sensed 
information storage section 184. 

The endpoint detecting section 154 detects comer points in the sensed 
projected image 220 based on the rectangular temporary sensed projected image 230 
(step S6). The technique of detecting the comer points will be described more 
10 concretely. 

The endpoint detecting section 154 basically searches each of the pixels on the 
respective sides of the temporary sensed projected image 230 starting from the 
respective comer points E, F, G and H on the temporary sensed projected image 230, 
and to detect a pixels each having its luminance value equal to or larger than a 

15 predetermined value (e.g., 2.0) as a comer point in the sensed projected image 220. 

If the endpoint detecting section 154 continuously detects two or more pixels 
having their ratios equal to or larger than 2.0 in the same searching direction, it may 
detect one pixel having the maximum ratio among a group of detected pixels or a first 
pixel having its ratio equal to or larger than 2.0, as a comer point. 

20 It the endpoint detecting section 154 is to search two different sides of the 

temporary sensed projected image 230 starting from each of the comer points thereof, 
the following technique may be taken. For example, when the pixels on the horizontal 
and downward sides (e.g., sides EF and EH in FIG. 2) of the temporary sensed projected 
image 230 starting from the leftward and upward comer point thereof (e.g., comer point 

25 E in FIG. 2) are to be searched and if a point having its ratio equal to or larger than 2.0 
is detected within a predetermined number of pixels in the rightward search starting 
from the leftward and upward comer point in the temporary sensed projected image 230, 
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the downward search starting from the aforementioned comer point may be continued 
until a pixel having its ratio less than 2.0 is detected, even though a point having its ratio 
equal to or larger than 2.0 has been detected, and detect a pixel just before the pixel 
having its ratio less than 2.0 as a comer point on the downward extending side. 
5 If a pixel having its ratio equal to or larger than a threshold is readily detected 

starting from a comer point in a search on any side, it is possible that one side of the 
sensed projected image 220 overlaps the other side of the temporary sensed projected 
image 230, for example, under the influence of quantization error since one comer point 
in the sensed projected image 220 is very near to one comer point in the temporary 
10 sensed projected image 230. 

Even in such case, the projector 20 can prevent such an overlap of sides and 
adequately detect the sensed projected image 220 by adopting the second search 
technique. 

However, if the search is performed only on the sides of the temporary sensed 
15 projected image 230 as described, it is possible that the endpoint detecting section 154 
has detected only three or less comer points. In such a case, the endpoint detecting 
section 154 also searches the interior of the temporary sensed projected image 230. 

FIG. 6 A is a diagram schematically showing a temporary sensed projected 
image 231 when three comer points have been detected; and FIG. 6B is a diagram 
20 schematically showing a temporary sensed projected image 232 when two comer points 
have been detected. 

If the endpoint detecting section 154 have detected only three or less comer 
points as shown in FIGs. 6A and 6B, it counts the number of pixels having their ratios 
less than 2.0 for each pixel line starting from four comers of a non-sensed projected 
25 image having its ratio less than 2.0 (i.e., a sensing area 210 other than the sensed 
projected image 220) toward the interior of the temporary in all the directions, and 
determines the ratio of counts for each set consisting of a predetermined pixel line and 
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another pixel line adjacent to the predetermined pixel line. The endpoint detecting 
section 154 then detects a comer point if the rates of change in this ratio and a ratio of 
counts for two pixel lines before the aforementioned pixel line exceed a predetermined 
value. 

5 The predetermined value used herein may be (an average ratio of counts in the 

same searching direction * 1.2) for example. In other words, the endpoint detecting 
section 154 may detect a comer point using a pixel line where its ratio of counts is 
separated off from the average value. 

For example, in the cases of FIGs. 6A and 6B, the rate of change for the ratio 
10 of counts will largely be varied since there is a large diflference in the number of pixels 
having its ratios of luminance value less than 2.0 between a pixel line wherein a comer 
point shown by arrow exists and another adjacent pixel line. Thus, the endpoint 
detecting section 154 can detect three points PI, P2 and P3 as comer points. 

In such a manner, the endpoint detecting section 154 detects four comer points 
15 in the sensed projected image 220. 

The pixel block image information generating section 156 converts sensed 
information for predetermined sensing areas of various colors such as white, red, green 
and blue into pixel block image information, based on the information of comer points 
from the endpoint detecting section 154 (step S7). 
20 The pixel block image information will be described. 

The pixel block image information is one that is provided by converting the 
ratio information in the sensed projected image 220 surrounded by the comer points 
detected by the endpoint detecting section 154 into information representing image 
signal values or ratio of luminance values for each pixel block that is formed by one or 
25 more pixels. This number of pixel blocks may one that is determined depending on the 
processing in any correction means such as the color non-uniformity correcting section 
130 or the like. 
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In an extreme example for easy understanding, a pixel block image converted 
from the sensed projected image 220 shown in FIG 2 may be as shown in FIG 7 if the 
spatial light modulator 192 includes three pixels in the vertical direction and four pixels 
in the horizontal direction. 
5 FIG 7 is a diagram schematically showing a pixel block image according to 

one embodiment of the present invention. 

As shovm in FIG 7, the sensed projected image 220 is partitioned into 1 2 pixel 
blocks, three blocks in the vertical direction and foxir blocks in the horizontal direction. 

The pixel block image information generating section 156 sets a group of pixel 
10 blocks depending on the processing in the correction means; computes the average 
image signal or luminance values in a group of pixels forming each pixel block, based 
on the sensed information of each calibration image stored in the sensed information 
storage section 184; generates pixel block image information showing the average 
limiinance value or the like for each pixel block; and stores the pixel block image 
15 information for each calibration image in the sensed information storage section 1 84. 

The image distortion correction computing section 162 detects the change of 
luminance value based on the pixel block image information for whit-colored 
calibration image stored in the sensed information storage section 184. The image 
distortion correction computing section 162 then detects a distortion in the projected 
20 image 12 based on the change of luminance value for the pixel block and computes the 
amount of image distortion correction. 

For example, if the luminance value on the left side of the projected image 12 
is relatively high, the image distortion correction computing section 162 can detect that 
the optical projection axis is out of the center of the projected image 12 in the rightward 
25 direction and that the projected image 12 is distorted into a trapezoidal shape having a 
left shorter side and a right longer side. 

The image distortion correcting section 112 corrects digital R-, G- and 
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B-signals to generate digital R-, G- and B-signals (R2, G2, B2) for correcting the image 
distortion, based on the amount of image distortion from the image distortion correction 
computing section 162 (step S8). 

Thus, the projector 20 can correct the image distortion. 
5 The color non-uniformity correction computing section 164 computes the 

amount of correction of input/output characteristic data for each pixel block, based on 
the pixel block image information. More particularly, the color non-uniformity 
correction computing section 164 may compute the amount of correction so that the 
inclination of a line representing the corrected input/output characteristic becomes one 
10 (1). 

The term "input/output characteristic data" used herein is intended to mean data 
that indicate the input/output characteristic and that also indicate the relationship 
between a brightness index value of an input signal (e.g., gray-scale value) and a 
brightness index value of an output signal (e.g., luminance, illuminance, or color 
15 information value). The brightness index value is a value providing an index of 
brightness and more particularly it may include luminance, illumination, color 
information value (such as digital R-signal value), gray-scale value and other values 
provided by deforming these values through normalization or the like. 

The color non-viniformity correcting section 130 corrects the input/output 
20 characteristic data to correct color non-uniformity in the image, based on the amount of 
color non-uniformity correction from the color non-xmiformity correction computing 
section 164, and corrects the R-, G- and B-signals (R3, G3, B3) into the R-, G- and 
B-signals (R4, G4, B4), based on the input/output characteristic data (step S9). 

Finally, the image projecting section 190 projects an image corrected for image 
25 distortion and color non-uniformity through the above-mentioned procedure (step SIO). 

As described, the projector 20 of this embodiment can precisely detect the four 
comers in the sensed projected image 220 in accordance with this embodiment and 
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therefore can precisely and effectively cause the coordinates of the sensed projected 
image 220 in the sensing area to be associated vsdth the coordinates of the spatial light 
modulator 192. 

According to this embodiment, furthermore, the projector 20 can adequately 
5 perform the correction and precisely and effectively cause the coordinates of the sensed 
projected image 220 in the sensing area to be associated with the coordinates of the 
spatial light modulator 192 by correcting the pixel blocks that are provided by 
partitioning the interior of the sensed projected image 220 in the sensing area 210 
depending on the processing in the correction means such as the color non-uniformity 
10 correcting section 130 or the like. 

The projector 20 can more effectively the image processing since the number 
of pixel blocks is less than the number of pixels in the original sensing area. 

According to this embodiment, the projector 20 can precisely and effectively 
detect the comer points in the sensed projected image 220 by using the ratio information 
15 provided from the computation of the ratio of image signal or luminance values for each 
pair of pixels in the sensing area 210, in comparison with the conventional technique of 
detecting the comer points through filtering. 

According to this embodiment, the projector 20 can more effectively make the 
image processing since it uses the ratios rather than the differential values and therefore 
20 does not require any change of judgmental criterion for the comer points in the sensed 
projected image 220 depending on the applied environment. 

According to this embodiment, the projector 20 detects the comer points using 
the different search method even though only three or less comer points can be detected 
when it is wanted to detect the comer points at the four comers in the sensed projected 
25 image 220. Thus, the projector 20 can precisely and effectively detect the four comer 
points in and cause the coordinates of the sensed projected image 220 in the sensing 
area to be precisely associated with the coordinates of the spatial light modulator 192, 
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even if the image is distorted into any one of various different shapes. 

According to this embodiment, the projector 20 can display a correct image 
since it can precisely detect the sensed projected image in the sensing area and since it 
can precisely correct the distortion and color non-uniformity in the image for each pixel 
5 block even if the sensed projected image is divided into pixel blocks for correcting 
image signals. 

According to this embodiment, the projector 20 can adequately correct color 
non-uniformity by time degradation or environmental impact (e.g., in a case where there 
is ambient light 80 or a case where the screen 10 is non-uniformly colored) since it can 
10 detect any differential color between the pixel blocks. 

Modifications 

Although the embodiment of the present invention has been described, the 

present invention is not limited to the embodiment. 
15 Although the embodiment has been described as to the projector 20 for 

correcting the image distortion and color non-uniformity, the present invention is 

similarly effective for any other correction. 

Although the above embodiment has been described as to the projector 20 for 

generating the ratio information by the use of the sensed information for white- and 
20 black-colored calibration images, a monochromatic green-colored calibration image 

may be projected and sensed in addition to the aforementioned calibration images. 

Based on sensed information for white and black colors and for green and black colors, 

two kinds of ratio information may be generated to compute the product set (AND) for 

two kinds of ratio information, thereby setting a true region as a region to be detected in 
25 the temporary sensed projected image 230. 

Consequently, the influence of any noise produced when the image is sensed 

can be reduced. Thus, the projector 20 can detect more precisely the temporary sensed 
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projected image 230 and sensed projected image 220. Although the embodiment has 

been described as to the projector 20 for detecting the sensed projected image 220 after 

the temporary sensed projected image 230 has been set, the sensed projected image 220 

may be detected directly from the sensed information. 
5 Although the embodiment has been described as to the projector 20 for 

detecting the temporary sensed projected image 230 by applying a pixel line as a 

predetermined pixel region, the temporary sensed projected image 230 may be detected 

by using one pixel, a plurality of pixels, a rectangular area configured by a plurality 

pixels or the like as a pixel region. 
10 Although comer points are used as edge points in this embodiment, points on 

sides other than the comer points of the sensed projected image 220 may be used 

instead of the comer points. 

Although the embodiment has been described as to the projector 20 for 

performing the search inward firom the outside of the temporary sensed projected image 
15 230, the search may be performed outward from the inside of the temporary sensed 

projected image 230. 

Although the embodiment has been described as to the image processing 

system moimted within the projector 20, the image processing system may be mounted 

in any image display device such as CRT (Cathode Ray Tube) other than the projector 
20 20. The projector 20 may be a projectors using DMD (Digital Micromirror Device) 

other than liquid crystal projector. DMD is a trademark possessed by the U. S. Texas 

Instruments. 

The function of the projector 20 may be accomplished solely by the projector 
20 or by a plurality of decentralized processing components (e.g., one projector and one 
25 PC). 

Although the embodiment has been described as to the sensor 60 separated 
from the projector 20, the projector 20 may include the function of the sensor 60. 
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